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ABSTRACT 

During the summer of 1968, a 6-veek inservice 
training institute was held at Duluth, Minnesota, for 45 experienced 
teachers from 13 secondary schools throughout the United States. 

These schools are part of a 17-school ES '70 Network banded together 
for the purpose of developing innovative curricula for the 1970‘s* 

The purpose of the institute was to prepare the 45 teachers in the 
following areas: writing behavioral objectives and developing 
"learning packages" to attain these objectives; gaining skill in 
group dynamics and developing more selfrunderstanding, openness, and 
ability to deal with affective phenomena: becoming acquainted with 
educational simulation; and gaining experience with a broadly-based 
humanities approach to learning. Significant gains were attained in 
measured ability to identify, evaluate, and develop behavioral 
objectives, and it measured attitudes toward teaching and students, 
in the direction of nore acceptance and understanding of students. 
Subjective, open-ended evaluation by enrollees and generally positive 
toward the institute, but also somewhat critical of the humanities 
and educational simulation aspects of the program. Based on critique 
by enrollees, changes are recommended for future institutes, and a 
reunion followup is advised to permit further evaluation and sharing 
of transfer values and experiences. (Author/RT) 
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SUMMAKT 



Individualized and personalized instruction has long been an 
ideal of American education. A relatively new approach to thiB 
ideal has been incorporated into the thinking of educational leaders 
in a group of 17 school systems banded together into what has been 
entitled tho ES >70 Network (educational systems for the 1970*8) . 

It is the intent of the member schools to develop a unified, inte- 
grated high school curriculum based on a systematic sequence of 
activities for which measurable behavioral objectives have been 
specified, and to develop assessment instruments to measure the 
attainment of those objectives, 

During tho summer of 1966, an in-service teacher training 
Institute was hold at IXiluth, Minnesota (one of the ES *70 member 
school districts), This six-week Institute brought together h$ 
experienced teachers from 13 of tho 17 ES *70 schools, for tho 
purposo of providing training in tho systoms approach to instruction, 
writing of bohavioral objectives, dovoloping of "loaming packages” 
to attain thoso objootivos, gaining understanding of and skill in 
group dynamics, developing more solf -understanding, openness, and 
ability to doal with affootlvo areas of student growth. In addi- 
tion, enrollocs wore to bocomo acquainted with educational simula- 
tion and gain oxporionces with a broadly based hu/nanitios approach 
to loaming, 

A full schodulo of activities was carried out in an unusual 
school sotting in which largo, open pods replaced classrooms, 
and in which about 100 high school students cerriod on their own 
summer school activities simultaneously, with much integration 
and overlap botwoon tho two groups. 

Significantly large gains wore found between pro- and post- 
testing in tho ability to identify, evaluate, and dovolop behavior- 
based measurable educational objectives. Significant gains woro 
also found from tho beginning to tho end In attitudes relating 
to teaching and students, indicating moro acceptance, openness, 
rospoot for students, and a mere rolaxod, solf-accoptlng attitude, 

Subjoctivo, open- ended evaluation of tho experience by tho 
enrolloos indicated a generally pcaitlvo fooling about tho 
Instituto, and an expression of satisfaction with nwny of tho 
loamlngs. Tho oxporioncos most often rated as holpful vero tho 
work with behavioral objootivos and tho group dynamics, particu- 
larly the aonaitivity training experienced lAto In tho Instituto 




ill- 



by two thirds of the members. Criticisms included concern for 
too loose a structure, too much use of films without directed 
follow-up activities, too much time in writing learning packages, 
lack of opportunity to "practice" use of the learning packages 
with the high school students, and, most of all, the educational 
simulation games. These games were critioised as being too 
expensive and time-consuming providing the wrong kinds of experiences 
(warlike in one instance), and/or being indications that hardware 
salesmen are becoming overly influential in education. 

One of the most frequently mentioned positive aspeots of the 
Institute was the living arrangements made possible for those who 
desired such arrangements, Dorms were opened at a local college, 
providing a fine e*,q>erience of living together and relating to one 
another in a congenial atmosphere. 

From the experience, it is recommended that* (a) cannon 
congenial living arrangements be available and if possible, manda- 
tory) (b) work on behavioral objectives be a core activity, but 
that once this activity is learned, the development of learning 
paokages be left to the most able and enthusiastic j (o) encounter- 
group experiences be provided early) (d) more structure be provided 
initially, for the comfort of the anxious) (o) fewer films be 
show with more deliberate instruction as their values) (f ) great 
oare be used in selecting educational simulation exercises, Indi- 
cating their values and showing preference for less expensive games) 
(g) follow-up activities be planned and funded, permitting enrollees 
to return after a year or so to share experiences and contribute to 
better planning of further Institutes, 



DfTHDDUCTION 



American education has long been attracted to tho ideal of 
individualized and personalized instruction for all students -- that 
is, arrangements that makes it possible for oach student to bo 
engaged in looming those things that trill bo of greatest benofit 
to him, at a rato appropriate for him, Whilo this ideal trill probably 
never be fully realized, progress toward it con be raado through the 
implement at ion and evaluation of innovative educational techniques. 

One approach toward this idoal is the ES >?0 Systems notwork of 17 
publio school systems throughout tho nation. 

Spooifically, tho ES >70 systems Approach is charaoterieod byi 

1, tho formulation of moasurablo objootivos and a systematic 
Boquonco of aotivitios designed to convorgo on tho boot methods for 
accomplishing those obJootJLvosj 

2, tho dovolopment of assessment instruments to ovoluato tho 
dogroo to vhioh tho objectives havo boon achiovod— and so, to vali- 
dato tho mothods and matoriols produced by tho program) 

3, tho utilization of tho assossraont instruments and a ”fiold 
environment” for ro vision and improvement j 

tit tho utilization and integration of sub J cot mattor exports, 
media spooialists, looming psychologists, and administrative por- 
sonnol within nulti-disoipllnary teems for planning and instruction. 

Each of tho 17 ES *70 schools has boon assigned a subjoct- 
raattor aroa in which it is cxpectod thoy \rill provido loodorship 
in dovoloping mothods for moving toward tho goals of tho project. 



Purposo 



During tho surmor of 1968, on IriStituto was hold at tho Chostor 
Park Elonontory school in tkiluth, Minnesota, for tho purposo of 
advancing tho goals of Projoct ES '70, Throo institutions assumed 
primary responsibility for planning and conducting tho Institute, 
Thoao woroi Tho University of Minnesota, Duluth) tho Collcgo of St, 
Sohola8tico (Duluth, Hinnosota) and tho Duluth Publio Schools, 

Specifically, tho six-week strrnor teachor education Ihstituto 
t/as doalgnod to contribute substantially to tho solution of a 
critical and imedioto ES '70 problem- -'tho problem of staff propara- 



tion, Beoause of the uniquo nature of the ES '70 program and the 
limited number of personnel in the country who have been exposed 
to its concepts, it is extremely Important that prospective ES *?0 
teachers roceivo adequate preparation so that they will perform 
well in their assignments* The purpose of the Institute was to 
provide in-service training for experienced ES *70 teachers in 
selected areas of instruction. These areas verot 

1* systems approach to instruction, including the develop- 
ment of behavioral objectives and the preparation of lesson plans 
to attain these objectives) 

2, introduction to educational simulation involving direct 
student participation in problem- so?. ving situations structured as 
games) 



3. development of group process skills) using Institute 
er.rollee8 themselves as the laboratory and including sensitivity 
training) 



h. emphasis on the integrative nature of humanities, using 
excellent films, books, and records, activities in tho classroom 
demonstration center and in the community, and focus on currently 
troublesome and relevant social issues) and 

5. continual emphasis on evaluation and feedback as to the 
effectiveness of teaching methods and, in the Institute, of the 
progrm in which enrollees participated* 

bach or the 17 schools in the ES '70 network was invited to 
send three participants. Other participants in the Ihstitute were 
the five faculty members, three demonstration teachers, and approx- 
imately 100 high school students who were registered in the Duluth 
Public Schools' surmer school program, 1 



*See Appendix B for names of Institute staff and demonstra- 
tion teachers. 



METHOD 



Subject 8 

A total of U5 teachers, 3h wale and 11 female, from 11 of the 
17 ES <70 schools attended the Institute* These enrollees came from 
all over the United States, from Portland, Oregon to Baltimore, from 
San Mateo, California, to Houston, Texas, to Quinoy, Massachusetts.!* 
The subjects were selected by their administrators on the basis of 
their experience, their potential contribution to the Institute 
and to their schools following the experience, and their availability 
to attend the Institute* Every effort was made to have a representa- 
tive sample which would include teachers from all disciplines, 
and this effort was lergely successful* Among the enrollees there 
were not only the standard high school subjeot matter teaohere 
(English, Soienco, Social Soionce) but also teachoi i of Russian, 
French, Physical Education, Industrial Arts, Stenography, Guidance — 
and even one teacher of the mentally rot&rdcd. (See Appendix A) 

Last minute approval of the Ihstituto oreatod some problems and 
rosultod in substitution for original selections in some cases] 
this may havo resulted in tho soleotion of somo participants who 
voro less uo31 prepared than tho original solootoos. 

Although about 100 high school students attendod classes at 
Chestor Park during the Institute and interacted froquortly in many 
ways with Institute onrolleos, they wero not subjocts of evaluation. 



Doeoriptlon of Program 

Tho program was conducted during the same period as tho 1966 
sooond^ > school surwior session program, and tho special facilities 
for individualized instruction which are availablo at Duluth's 
Cheater Park Elementary School wore used to house the Institute 
and that portion of the Duluth secondary school sumor program 
which sorved as the demonstration contor for educational simulation, 
integrated humanities activities, and othor practlcum aepoots of 
the Institute's activities* This arrangement made it possible for 
tho Instituto members to observe various kinds of activities and 
also for oaoh to dovolep and to a limited axtcmt, tty out with 
students his own strategies In tho areas undor study in tho Instituto, 

Tho Chostor Park School includes throo teaching pods and ono 
instructional materials contor. Each pod is tho slzo of four 
traditional classrooms. There arc no Interior wells within any 
ped, nor between tho pods and the instructional materials center. 



^See Appendix A for the list of schools and the enrollees 
from each. 



The entire area is carpeted throughout! and the overall Impression 
is one of large flowing spaces* Also included in the facilities 
are an office, a work center, an audio-visual workroom, a teacher's 
workroom, and conference areas* This was the setting within which 
the Institute was oonduoted* 

Early in the Institute the major emphasis was on the theory 
and praotice of writing behavioral objectives and developing teach- 
ing units and packets of materials designed to attain these objec- 
tives. During the rest of the time, each emollee had much time 
to practice and develop these skills, while also developing appro- 
priate units within his area of competence* While doing this the 
enrollee bad available to him the advi^o and help of Institute 
staff members, as well as meny opportunities to consult and work 
together with other enrollees* Ho had available to him the library 
facilities not only of the Institute but also Of the two oollegea 
in Duluth* In this manner, explicit provisions were made for the 
procedures of tho Institute itself to be consistent with the goels 
of individualized instruction, so that theory and praotice would be 
interwoven* A largo numbor of excellent packets of teaching materials 
were produced, and then reproduced and provided to each of the 
Institute members* After the early atagoa, this work was all indi- 
vidualized and fitted in with other ongoing activities. 

> 

The humanities program was baaed on a series of 28 top quality 
art films from all over tho world, 1 a varioty of taped and recorded 
music, and a number of paper backs* These materials were the basis 
for discussion, activity projeots, coranunity surveys, jtudies of 
local minority groap problems, and many other Activities involving 
both institute members and high school students* A number of learn- 
ing units based on these materials and activities were also pre- 
pared by institute members* The basic purpose of these activities 
was to challenge Institute enroll ees and high school students to 
encounter and ponder Vie basic hvxnan questions of our day, to sesk 
to fir.i tbem8olves ar.d their own meanings and to express these 
artistically as they felt able to do so* 

*Some idea of the breadth and scope of films used in the 
Institute can be gained from the following titles, seleoted from 
the list of 28 films show, Rocoo and His Brothers, Yojiinbo, 

Ky Heme is Ivan, Marshall McLuhan, Tho Seventh Seal, Ugetsu, 

Landscape of the Body, Visit In Picasso, Nothing But a Man, Vforaan 
in the Dines* It was, of o;urse, Impossible for all enrollees to 
see all of these films— at any given time some would be engaged in 
various other activities while othere viewed and then discussed films* 

- U - 




Tho basic form of educational simulation was the computor- 
supportod Ihtor-Nation Simulation exercise, INS is a functional 
system consisting of tho main dements of international politics. 

It enablos students to manipulate in a simplified vay some of tho 
key faotors in international relations, and it doos so in a manner 
that pormits tho students to formulate their own strategies for 
the oountrios involved. Tho strategies of tho various countries 
oro fod into tho computer which calculates probabilities of verious 
o vents and outcomes and reports what happened, after which more 
stratogio moves ore mode. It proved a challenging and oxciting 
game for tho high sohool students, and ono that roquirod thorn to 
loam much about tho countries involvod. In playing tho gomo they 
also sot up "nations" in parts of tho pods, with artifacts, music, 
and othor oultvral displays. The enrolloos wore obsorvors and 
occasionally partioipated in aspects of planning, and in some 
"spin-off" aotivitioo which resulted from tho gomo, 

Tho onrolleo8 met in small groups to study and to exporienco 
tho dynamics of groups, using various materials rolnting to group 
and olass interaction and using their o\n group as a laboratory. 

Out of those snail group activities thoro grow an intorost in sensi- 
tivity training or encounter groups, and toward tho ond of tho 
Instituto, threo T-groups vero formed with ono mombor of tho staff 
as facilitator. These groups mot for two mini-marathon Bossions 
at a lnko cabin rotreat. Thirty- ono of tho onrolloos partioipatod 
in thoso activities, dosigned to incroaso solf -awareness, oponnoss, 
sensitivity to othors and ability to doal honestly with foolings, 

A typical day might include working on porfomanco objectives 
and looming activities in ono>c own area, sooing a film and 
participating in a discussion of tho film, (sometimos in groups 
including high school studonts), participating in small group 
process, obsorring tho I.N.S. activity and participating in ono 
of its spinoff activities such as a projoct on the generation gap 
or on raco relations. On ether days there might bo visiting 
spoakore, panel discussions, student presentations, visits to tho 
Outhrlo thoator, showing of slldos from Turkey by on enrolloo 
who had boon in tho Peace Corps, soesions on how to nako films, 
dlsouasion of unusual looming packagos by onrolloos who had com- 
pleted suoh ftealgmcnts, visits to local schools to observe indi- 
vidualised instruction, and many othor activities generated during 
tho Instituto* Ono such activity was a brief encounter by enrolloos 
with anothor educational simulation gomo entitled "Impact," 

Six gradaato-lovol quarter credits wore awarded, (as desired) 
for attendance at tho Institute, Theso orodits wore granted from 
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tho University of MLnnosotn, Duluth, and assigned to two courses 
entitled EdCI 112, Progra mmed Instruction in tho Classro om (3 or.) 
and EPsy 125, Group Dynmics in Education (3 or.). 



Evaluatlvo Instruments 



It was hypothesized that tho program of training and the 
typos of activities would produco changos in attitudes, knovlodgos 
and skills of tho qnrolloos. In addition to rogular foodback 
fran onrolloos and ongoing evaluation and changos in tho format 
resulting from such foodback, sovoral measurements woro mado nt 
the boginning and ond of tho Instituto, Tho following instruments 
wore administorod on tho first day and again on tho last day of 
tho Institute: 

1. Tho Minnesota Teachor Attitudo Inventory (MTAI) . This 
is a popular and vidoly usod instrument for tho measurement of 
toachor att.itudos, A high scoro on this l^O-item Likort-typo 
instrument presumably indicates n warm, occopting attitudo toward 
pupils, and an understanding of and rospoct for children. Con- 
versely a low scoro would indicato a rigid, somewhat hostilo and 
inseouro toachor, Tho scalo has accoptablo reliability, and a 
numbor of studios tond to support tho claim that tho tost measuros 
tho attitudinal dimonsion doscribod abovo, 

2, Modified 80~itom Attitudo Inventory. From tho total of 
lf>0 items on tho MTAI, a team of educators involved in individua- 
lized instruction was asked to soloct items which they folt apply 
with particular emphasis to individualized instruction using tho 
bohavioral object ivos approach. Thoso toachors agrood on oighty 
such items, which woro scored separately, using tho standard MTAI 
scoring key. These 80 items form a speoial modified MTAI. It 

was assumed that tho itoms on which thoso toachors (all oxporioncod 
in individualized instruction in tho Daluth system) could agroo 
would provide a roliablo index of attitude chango as it rolatos 
to individualizing tho instructional program. No effort was mado 
to valtdato this assumption; thoroforo rosults reported from this 
subtost can only bo considered as suggostivo. (Soo Appendix C 
for tho modified MTAI). 

3» Educational Objoctivos Quiz. This is a short tost do- 
volopod by Popham and Bakor, designed to moasuro tho subject’s 
skill in writing, and in identifying properly written bohavioral 
objoctivos. Tho validity of this quiz can best bo determined by 
examination of tho content of tho items. (Soo Appondix D) 



O 
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h* Instructional Objoctivos Proferonco List (IOPL) • This 
is a list of instructional objoctivos ivhich tho subject is roquirod 
to rato as axcollont to poor. This list was dovolopod by W. Janos 
Popham for uso in training programs in writing behavioral objoctivos. 
Again, validity is assumod from tho contonts of tho list, and agroo- 
mont among oxporioncod writors of bohavioral objoctivos. (Soo 
Appendix E for tho IOPL,) 

Methods-! ochniquos Quostionnairo, During tho planning 
stago, tho Instituto staff expressed n dosiro to givo onrolloos 
an opportunity to doscribo their own approaches to planning and 
conducting o teaching unit. It was felt that changes in tho way 
this opon-endod assigunent was carriod out would provi.de a bohavioral 
moasuro of learning during tho Instituto, In order to accomplish 
this purposo, tho following instructions woro given to each cnrolleo: 

Givon a class of 20-30 students in a subject of your choico, 
for which you have suffioiont timo to preparo, doscribo the 
methods, techniques, and rosourcos you would uso. Include 
any othor aspects of preparation and operation for tho class. 

This was givon to onrolloos upon arrival and turned in a day 
or two later. It was givon again on tho last day of tho Institute, 
Staff members dovolopod a scoring mothod which included the following 
aroas: specification of instructional objoctivos, purposos, tests 

rolativo to objoctivos, and protesting* diversification of lessons 
and aotivitios so as to covor wido range of abilities and interests* 
pacing* interaction of toacher- students and students- students* 
rolovonco of matoriol to students including opportunity to partic- 
ipate in selecting objoctivos. 

In addition to these instruments, all onrollees filled out free- 
rosponso reactions to all phases of the Instituto. 



Method of Analysis . 

Tho z-tost for significance of difference of means of pre- 
and posttosts was used with tho MTAI, tho modified MTAI, tho 
Educational Objoctivos Quiz, and tho Instructional Objectives 
Proforonco last using tho .01 level of significance for rejoction 
of tho null hypothesis. For each test the null hypothesis was 
that tost scaros do not chango when subjects are exposed to tho 
kinds of learning experiences dosoribod in this report. 
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It wag intended that the sane test he applied to the raethods- 
teohniques questionnaire but this did not turn out to be feasible 
(See next section) 



RESULTS 



In Tables 1-b are summarised the results of the pro- aid post- 
tests for the Minnesota Teacher Attitude Inventory, modified MTAI, 
Educational Objectives Quia, and Instructional Objectives Prefer- 
ence List* Each table includes the pre- and posttest means, the 
mean difference (M^), the number of subjects (N), the standard 
error of the mean difference (Sw), and the Z-value. The number of 
subjects varies and is less than the total enrollment beoause seme 
left early and one or two did not take all of the tests* 



Table 1 

Minnesota Teacher Attitude Inventory 
N-37 





Testing 


Mean 


«D 


% 


Z -Value 




Pre-test 

Post-test 


51.30 

63*35 


12.05 


2.63 


U.58** 




#»sig* 


at .001 level 











The null hypothesis is rejected) the results clearly indicate 
significantly higher scores in the post- test. 



Table 2 

Modified 60-Item X> 

N-38 





Testing 


Mean 


«D 


% 


Z-Value 


Pre-test 

Post-test 


26.32 

33.92 


7.61 


1.51 


5.0l*K# 



**£&gnif leant at .dol level 



The null hypothesis Is rejected) again the results indicate sig- 
nificantly higher post-test scores. 
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Table 3 

Educational Objectives Quiz (BOQl 
N*»39 





Testing 


Mean 


”0 




Z-value 


Pre-test 


70.79 












9,i*9 


2.U9 


3.81** 


Post-test 


80.39 









•^Significant at ,OGl level 



The null hypothesis is rejected; post- test scores are signiff 
cantly higher. 

Table h 

Instructional Objective# Preference List (IOPD) 

N=38 



Testing 


Mean 


«D 


Sg Z-value 


Pre-test 

Post-test 


7S.63 

83.71 


8,08 


1.67 h.8 1*** 



^Significant at ,CX)1 level 



The null hypothesis is rejected; post-test scores are signifi- 
cantly higher. 



The scoring method used in attempting to arrive at a single score 
on the Methods- Techniques questionnaire turned out to be of slightly 
questionable reliability, so that quantitative analysis of the results 
might have been rather misleading. Visual examination of the results 
left the impression that there was seme "improvement" in the area of 
specifying instructional objectives, but either no change or reduction 
of scores in the other areas, Thia matter is discussed further in 
the next section, in which enrollees reactions are also summarized. 



DISCUSSION 



A large part of the work of the Institute centered around the 
preparation of precise, measurable behavioral objectives, and of 
preparing "packages" of materials designed to attain these objectives. 
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Following initial lectures, examples, demonstrations, each enrollee 
began working on his own, aided and guided by staff members. This 
being so, it would Indeed have been surprising had the enrollees 
not during this 6-week period become better able to write and to 
identify good behavioral objectives. At the same time, it should 
be borne in mind that these were seleoted experienced teachers from 
schools in the ES »70 network, had already had exposure to this 
approach to teaching, and could be expected to do well on th>i pre- 
tests measuring this skill* 

As indicated in Tables 3 and J*, this group did indeed do well 
in the two pre-tests measuring the ability to recognise, evaluate, 
and write behavioral objectives (the EDQ and the lO 1 ^) j yet their 
scores increased markedly during the Institute, It mey be concluded 
with considerable confidence that experienced teachers coming to 
suoh an Institute as described in this report can be expected to 
improve their knowledge and skills in dealing with behavioral 
objectives during the course of the Institute. In this regard, then, 
dearly the Institute successfully attained its goals as measured 
in the behavior of the enrollees. 

As indicated earlier, the open-ended assignment entitled 
"Methods-Techniques Questionnaire" did not work out well. The pre- 
test was given to enrollees when they came, and they were permitted 
to take it out and work on it for about two days. The result was 
that they wrote at great length and turned in very painstaking, 
serious products. At the end of the Institute the same form was 
given out on the last day. Several had already left, and the rest 
were in no mood to take a great amount of time on the instrument. 

Thus the two testings were not comparable. In addition, the inter- 
Judge reliability of the scoring among the three Institute staff 
members who attempted to score was somewhat less than desirable. 

But the general impression of all three Judges was that because of 
the testing error the second testing showed rather brief and care- 
less replies and would therefore have been judged to be of poorer 
quality than the first testing, except in the area of specification 
of instructional objectives. This last observation tends to corrob- 
orate the results of the two tests. 

The question of what might happen to the attitudes of enrollees 
was a much more open one. No direct attempts were made to change 
attitudes toward children, toward teaching, toward individualized 
instruction. However, the small-group experiences emphasizing group 
dynamics and the many discussions centering on human problems and 
concerns as showi in films and found in readings and spin-off projects 
from the Intemation Simulation exercise might be expected to have 
some impact on attitudes. In addition, the experience of sensitivity 
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training, while focussing only on the "here-and-now" and the parti- 
cipants themselves, was expected to influence attitudes in the 
direction of more openness, more non judgmental acceptance of others 
and differences in others, as well as self-acceptance. If such 
changes occurred, it seems reasonable to assume that they would 
result in changed attitudes toward teaching and toward children. 

As indicated by the results reported in Tables 1 and 2, the increase 
In MTAI scores tends to support this assumption. 

Even more tentative is the finding that attitudes beoome more 
favorable with regard to issues especially pertinent to individu- 
alized instruction, as indicated fcy the modified 80-ltem attitude 
scale. The use of this modified scale in this way needs further 
study and validation. 

In addition., each enrollee was asked to fill out a form indi- 
cating his reaction to each phase of the Institute, 1 As might be 
expected, practically every aspect of the summer progran was selected 
by someone as most valuable and by someone else as least valuable. 
However, some things did stand out. A large number of enrollees 
felt that the work on behavioral objectives was very valuable, and 
a lesser number found the work on "packages" of Instructional materials 
to be of great value, tti the other hand, a fair number of others were 
disappointed or even upset that so much emphasis was put on develop- 
ment of the "packages, " a task they felt they could have worked on at 
home later. In this connection it should be noted that a few enrollees 
completed no instructional units at all, some did one or two inferior 
ones, and a few made up brilliantly conceived and really outstanding 
units. 

Although only 31 of the enrollees actually took part in the 
sensitivity training, this aspect of the Institute was selected more 
frequently perhaps than any other as the outstanding experience of 
the Institute, A number commented specifically on the value of 
getting to know themselves better and/or developing sensitivity and 
skills which they felt would make them better teachers. Such reactions 
are known to be fairly typical for participants in encounter groups , 
and reflect to some extent the rather dramatic quality of the experience, 

must be noted here that all evaluative materials from the 
Institute were assigned by Mr, Thorvald Esbenson to a Duluth teacher 
for analysis and summary. After holding them for a considerable 
time without working on them, he left the system for another part 
of the country and it has proved impossible to recover the enrollees 
reaction-statements from him, Thus, this part of the report is 
based largely on recollections from having studied these statements 
earlier, and from some later letters received from participants or 
their superiors. 
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as contrasted to the more mundane experience of working on be- 
havioral objectives, for example. Nevertheless, it would seem 
apparent that such experiences are well worth considering in 
relatively short workshops or institutes where it is of value 
to develop a warm, accepting relationship among participants. 

The opportunity to live and work with teachers from all parts 
of the country was frequently mentioned as one of the real values 
of the Institute, In this connection, the living facilities at 
St, Scholastica, and the vira hospitality at that school, were 
frequently and very favorably mentioned. 

The large number of films produced a very mixed reaction, 
but on the whole the enroll oes appeared to feel that too many 
films were showi with not enough structuring so that possible 
educational values may have been realized. In general the 
humanities aspects of the Institute were criticized rather 
extensively as being r.ot well organized and of dubious value. 

It should be noted that the value of such activities as seeing 
and discussing films dealing with human problems is less easy to 
pin down than is the value of skill-oriented activities. The 
very fact that a number of people were really disturbed by some 
films indicates that the films wore having an impact} furthermore, 
there were animated and sometimes very heated discussions following 
the films, as might be expected since many raise questions about 
values hold dear and probably never previously questioned by many 
members of the institute. Still, the criticism is well taken that 
there were too many films and not enough development of the themes 
or questions raised. 

Another aspect of the Institute that came in for considerable 
criticism was the lack of structure. Many felt that a mors rigid 
structure with specific assigned work and strict accountability for 
acccmp.licliing that work would have henh an Improvement. It seems 
evident that people who need such structure would have produced 
m >ro than they did under ths rather relaxed conditions requiring 
a good deal of self-direction and motivation. Other enrollees, 
howevor, expressed appreciation for the opportunity to shape their 
own behaviors in directions they felt most useful, and expressed 
appreciation for the "flexibility* 1 of the staff. 

The visiting lecturers too received mixed evaluation, with many 
enrollees expressing the feeling that they did not contribute a 
great deal, while a number of others rated their contributions very 
highly. 
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Several understood beforehand that they would have more 
opportunity then they did to work with the summer school students 
in the role of teachers, "trying out" their learning units on 
these students. These teachers were disappointed in having 
relatively little opportunity to do so. All were able, however, 
to relate to the students in mow informal ways, such as in work- 
ing together on a humanities project, community survey, or the 
like; and many enrollees expressed appreciation of the opportunity 
to have this kind of olose relationships with students in a non- 
teaching role. 

Perhaps the sharpest criticism was diroot ed at the simulation 
games. One such game, "impaot," was introduced to the enrollees 
early in the Institute, but there was so much opposition to con- 
tinuing the game that it was aborted. After that they observed 
the Inter-nation simulation, and many of them were critical of 
that for various reasons. Some felt that it was too expensive 
and elaborate; some doubted its educational value; some were 
alarmed to observe that it soon appeared to deteriorate into a 
big war game played with considerable enthusiasm until the earth 
was blown up. Some viewed with alarm the possibility that hard- 
ware merchants were moving in rapidly with the increase in Federal 
funds for education, and might begin to take over educational tasks 
and prerogatives for private profit. 

Finally, there were a number of comments indicating that more 
should have been done to clarify the goals of ES *70 and means for 
attaining these goals. 



CONCLUSIONS, RE COMMENT AIONS 

This six-week s\mmer Institute at Duluth, Minnesota, participated 
in by 1*5 teachers from 13 of the 1? schools in the ES '70 Network, 
was designed to attain five major objectives. The evidence indicates 
that it was successful in developing knowledge of and skills in recog- 
nizing, evaluating and writing behavioral objectives and in developing 
teaching materials and lesson plans to reach these objectives. It also 
appears to have been successful in developing group process skills 
and attit’ides of greater openness to and acceptance of self and 
others, with somo resulting changes Jn attitudes toward teaching and 
pupils. 

The evidence i3 less positive with regard to the success of the 
Institute in the areas of Humanities education, or at least the en- 
rollees did not leave with a strong sense of ways in which they them- 



selves night put this education to work. Still less certain was 
the value of the introduction to educational simulation, ftince 
it left a majority of enrollees with negative attitudes about 
that kind of educational experience. Finally, constant evaluation 
and feedback were attempted during the Institute, and also provided 
to the enrollees as they progressed with their behavior-based 
units. The foedback from enrollees resulted in some changes in 
the course of the Institute as it progressed. Whether these 
changes were Improvements is difficult to say. 

Based upon the evaluation, observations, and reactions of 
Institute enrollees, the following recommendations and suggestions 
are made for future ES ’70 Institutes. 

1. Having participants from all sections of the country and 
all of tho disciplines or learning areas covered in high schools, 
and having minority group members included, makes for a very en- 
riching, rewarding experience for all concerned. 

2. If it can possibly be arranged. Institute enrollees should 
be housed in seme common aroaj this was rated a 3 very desirable by 
both those who stayed at St. Scholastica and many of those who did 
not. 



3. Study and discussion of, and practice in writing behavioral 
objectives and producing materials for attaining such objectives 
is an important activity which can be carried out very successfully. 

li. However, there are such tremendous individual differences 
in the ^kill and enthusiasm which teachers bring to this task that 
it kou eem unreasonable and inefficient to require all teachers 
*o engage n such endeavors (beyond tho point where they can under- 
stand how t ? dc It and appreciate its value) , It would seem more 
profitable to assign this task to those teachers or groups of 
teachers in each discipline who can do it well and who want to do 
it. 



5>. Institutes of thi3 type should build in encounter-group 
experiences very early In the Institute, so as to attain the great- 
est possible benefits to all enrollees during the time of the Insti- 
tute, and so as to create a warm, congenial, relaxed atmosphere in 
vdsich to carry out the rest of the Institute. Participation in such 
activities must be voluntary. 

6. Fairly well- structured organization and management, especi- 
ally during early sessions, would do much to relieve anxiety in 
teachers who have become accustomed to expecting and to providing 
organized learning situations. Such structure could probably be 
relaxed later in the Institute with little difficulty if suggestion 



no. U (early T-grouplng) were followed. 

7. Fewer films should probably be used with more carefully 
planned ways of drawing value from them and of making sure that 
the enrollees actually recognize the value and plan ways to use 
such films offectivelyj perhaps one or two excellent films per 
week would suffice. 

8. If educational simulation devices are used, more explicit 
and specific attempts should be made to indicate their educational 
purpose and values. It seems of dubious value to detao:uitrate simu- 
lation requiring extremely expensive hardware as well as constant 
supervision by representatives of the companies providing the 
hardware, a ■'nee this is not only impracticable for most school 
districts i the foreseeable future but ilso tends to shift the 
determination of education*! wothode and objectives into the hands 
of private enterprenours ■ 

9. Future Institutes should be funded early enough so that 
more careful pluming, based on precise information as to extent 

and type of funding con b8 made, so that it will not be •i.ecessary 
to work without such information until the very week of the begin- 
ning of the Institute, as was true in the present instance. 

10. Future Institutes should be funded adequately enough so 
that the enrollees, or at least a representative cross-section of 
them, can be called back after a year in order to report how they 
have made use of their learnings, how they now view the experience, 
and what changes they would recommend. Such a follow-up meeting, 
hold fer a week or more, would permit cross-fertilization of ideas 
carried over from the Institute to one or more schools, rekindle 
enthusiasms which may havo waned during the year because of other 
demands and presses, and allow for planning better institutes in 
the future. 

3h summary, both the measurement data and the basically enthu- 
siastic and positive comments of enrollees provide evidence that 
this Institute achieved its major goals, but that with the knowledge 
now at hand a numbsr of mistakes could be avoided in the future, 
and a considerably better and more effective Institute planned. 
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Appendix A 

Schools Represented in the Institute, Personnel 
from Bach School, and Major Teaching Area of Each 

San Mateo, California 

Orimes, Robert D,~ Ehglish 
Qvosden, Milo— Sooial Studies, omani ties 
Mast, Lloyd J» — Business Education, Shorthand 
McLean, Robert A. Jr. — Humanities 

Boulder, Colorado 

Cohen, Philip E. — Social Studies, American history 

Cox, Gary 0.— Music 

Krom, Mary L.~ Mathematics 

Siktars, MaiJ a— Mathematics 

Watkins, Robert— Science (Chemistry) 

Jackson, Kentucky 

Frasure, Malcom H.— Biology 
Rutherford, Darrel — English 

Quinoy, Massachusetts 

Forsyth, Richard W.— Automotive and related 
Fowke, Ronald — Guidance & counseling 

Monroe, Michigan 

McEven, William — Mathematics 
McMillan, Kenneth — English 

Duluth, Minnesota 

Carlscn. June A. --Ehglish 
Golen, Leonard L. — English, humanities 
Heikkinen, Marvin E. — Social Studies 
Muipliy, Mhry C. — Social studies 
Scderberg, Don aid- -Humanities 
Turner, Robert— Spanish 

Baltimore, Maryland 

Press, Joseph P. — Electronics 
Williams, Mamie B.— Business, stenography 

Willlngboro, New Jersey 

Brandau, George C.— Vice Principal 
Burrows, Arthur— Science 
Reilly, Patrick T.~ Ehglish 
Rocco, Harry — Mathematics 
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Mamaroneck, New York 

Rogovwky, Eavid— Social Studies 
Rogowsky, Janet— French and Russian 

Mined a, New York 

Debski, Merrill- -Music 
Fishkin, Rhoda K. — English, humanities 
Weston, JesB--Science 
Young, Frank D.- -Mathematics 

Portland,. Oregon 

Fuller, Robert H. --Industrial arts, metal 

Hedeen, Lauren—Engliah 
Hill, Glenn B. --Business Education 
O'Brien, Joseph- -Mathematics 
Schwab, Carl— Social studios 

Houston, Texas 

Bryant, Faye B,— Guidance, counseling 
Burney, Paraloe- -English 
Gray, Naurita— Mathematics 
dnith, Roland— Prin. , soc. studies 

San Antonio, Texas 

Bryant, Bessie L. --Mentally retarded 
Troy, Clifford- -Physical education 
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Appendix B -- Institute Staff 



FacuKy of the Institute 

1* Sister Margaret James* O.S.B.* College of St« 
Scholastics, Duluth, Coordinator of Projeot 
Criterion 

2* Dr* Annas W, Tbmminen* Professor and Head, Depart- 
ment of Psychology, University of Minnesota* Duluth 

3# Charles L* Jenks, Director of Research & Development* 
Reed Union School Distriot* Belvedere-Tiburon* 
California 

ti* George Ramos, Consultant for Special Projeots 

(Humanities), Duluth Public Schools, Duluth* Minnesota 

J>* Roger B* Tunks, Assistant High School Principal* 
Portland* Oregon 

Demonstration Teachers 

1* Brantly Bublitz, East High School* Duluth 

2* Dale Koch* Chester Park Elementary School, Duluth 

3* Robert Zbasnlk* Washington Junior High School, ttaluth 
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Appendix C 

Modified 80 - Item M.T.A.I. 

I* (3) Minor disciplinary situations should sometimes be turned 
into Jokes. 

2. (5) Teaching never gets monotonous, 

3. (9) A child should be encouraged to keep his likes and dislikes 
to himself. 

lj. (10) It sometimes does a child good co be criticized in the 
presence of other pupils, 

5. (13) The first lesson a child needs to learn is to obey the 
teaoher without hesitation, 

6. (Hi) Young people are difficult to understand these days, 

5. (15) There is too great an emphasis upon "keeping order" in 

the classroom. 

8. (16) A pupil's failure is seldom the fault of the teacher. 

9. (17) There are times when a teacher cannot be blamed for 
losing patience with a pupil. 

10. (19) Pupils have it too easy in the modem school. 

11. (20) A teacher should not be expected to burden himself with 
a pupil' 8 problems. 

12. (21) Pupils expect too much help from the teacher in getting 
their lessons. 

13. (23) Most pupils do not make an adequate effort to prepare 
their lessons. 

Hi. (2ii) Too meny children nowadays are allowed to have their ovn 
way. 

15* (25) Children's wants are Just as important as those of an adult* 

16. (26) The teacher is usually to blene when pupils fail to follow 

directions. 
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17# (29) Children have a natural tendency to be unruly, 

18, (31) Some children a9k too many questions. 

19# (3b) A teacher should never acknowledge his Ignorance of a 

topic In the presence of his pupils, 

20. (35) Discipline in the modem school is not as strict as it 
should be. 

21. (36) Most pupils lack productive Imagination. 

22. (37) Standards of work should vary with the pupil., 

23. (38) The majority of children take their responsibilities 
seriously, 

2b. (39) To maintain good discipline In the olassroom a teacher 

. needs to be "hard-boiled," 

25. (bO) Success is more motivating than failure. 

26. (b2) Every pupil in the sixth grade should have sixth grade 
reading ability. 

27. (b3) A good motivating device is the critical comparison of 
a pupil's work with that of other pupils. 

28. (b5) Course grades should never be lowered as punishment. 

29. (b6) More "old-fashioned whippings" are needed today. 

30. (b7) The child must learn that "teacher knows best," 

31. (b8) Increased freedom in the olassroom creates confusion, 

32. (50) Teachers should exeroise more authority over their pupils 
than they do. 

33. (52) The low achieve.' probably is not working hard enough and 
applying himself, 

3b. (53) There Is too much eephasis on grading, 

35. (57) Many teachers are not severe enough in their dealings 
with pupils, 

36. (61) Children are usuclly too sociable In the classroom, 
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37. (62) Most pupils are resourceful when left on their own. 

38. (63) Too much nonsense goes on in many classrooms these days. 

3 9» (66) I upile who fail to prepare their lessons daily should be 

kept after school to make this preparation. 

1*0. ( 69 ) Assigning additional school work is often an effective 

means of punishment. 

1*1. (71) Children should be allowed more freedom in their execution 

of learning activities. 

1*2. (71*) Pupils usually are not qualified to select their oxi 

topics for themes and reports. 

1*3. (76) There is too much leniency today in the handling of 

children. 

1*1*. (78) The whims and impulsive desires of children are usually 

worthy of attention. 

1*5. (79) Children usually have a hard time following instructions. 

1*6. (80) Childlen nowadays are allowed too much freedom in school. 

1*7. (81) All children should start to read by the age of seven. 

1*8. ( 82 ) Universal promotion of pupils lowers achievement standards. 

1*9. (86) If a child wants to speak or to leave his seat during the 

olass period, he bhould always get permission from the teacher. 

50. (93) Children should be given more freedom in the classroom 
than they usually get. 

51. ( 96 ) Pupils are usually alow to "catch on" to new material. 

52. (98) Pupils can be very boring at times. 

93. (100) Children must be told exactly what to do and how to do it. 

51*. (102) Whispering should not be tolerated. 

55. (105) A teacher should never leave the class to its o>n menage- 

men t. 
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J>6, (106) A teaoher should not be expected to do more work Vhan 

he Is paid for* 

*>7. (108) ’lack of application* is probably one of the most frequent 

causes of failure* 

*>8. (109) Young people' nowadays are too frivolous. 

£9* (110) As a rule teachers are too lenient with their pupils* 

60* (ill) Slow pupils certainly try one's patience. 

61. (lib) Children usually will not think for themselves. 

62. (ll£) Classroom rules and regulations must be considered in- 
violable* 

63. (116) Most pupils have too easy a time of it and do not learn to 
do real work. 

6b* (120) There is usually one best way to do school work which all 

pupils should follow. 

6$. (121) It isn't practicable to base school work upon children's 

Interests. 

66* (123) Children that cannot meet the school standards should be 

dropped* 

67* (12b) Children are usually too inquisitive* 

68* (126) Children today are given too much freedom. 

69 * (127) One should be able to get along with almost any child* 

70. (128) Children are not nature enough to make their om decisions* 

71. (130) Children will think for themselves if permitted. 

72. (13b) Most pupils are not Interested in learning* 

73* (136) A pupil should always be fully aware of what is eopected 

of him. 

7b • (Ujl) Teachers should not expect pupils to like them* 




23 



i 



7$. (Uf2) Children act more civilised than do raary adults. 

76. (Ut3) Aggressive children require the nost attention. 

77. (ll»I>) Young people today are just as good as those of the 
past generation. 

78. (11|6) Keeping discipline is not the problem that many teachers 
claln it to be. 

7?. (147) A pupil has the right to disagree openly with his 

teachers. 

80. (149) One should not expect pupils to enjoy school. 
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EDUCATIONAL OBJECTIVES QUIZ NAME 

Part I, Please complete this section before you look at Part EC • 

If necessary, change each objective below so that it is 
stated in terns of student behavior, (Either make written 
modifications of the objective or write out a new objoctive,) 



A, At the end of the course, the student will understand funda- 
mental concepts of Biology, 



B, The olass will learn about labor management relations. 
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EDUCATIONAL OBJECTIVES QUIZ 



NAME 



Part n. Place an X before any of the following Instructional 
objectives which are properly stated. 

_ _____ The student will grasp the significance of the Treaty of 

Versailles. 

The student will have an attitude favorable to English 
grammar indicated by his response to a questionnaire. 

The student will know six *erbs. 



The student will leam the names of the comon tools in 
wood shop. 

The teacher will list three major causes of the Civil War 
on the chalkboard. 

The student will know the important battles of World War I* 



JThe student will prefer cooking to sewing* 

JThe student will be able to correctly thread a iluwing 
“machine. 

The student will pay attention as the teacher demonstrates 
‘the use of the lathe. 

The student will be able to develop a sense of the cultural 
unity of man. 

The student will list and desoribe the themes of four of 
‘dhelley -a poems. 

The child will develop interest in leisure sports. 

The student will give indications of a desire to leam 
more history by volunteering to present an extra oral report. 

The teacher will describe with understanding five concepts 
‘treated in the text. 

The student will correctly solve all of the story problems 
presented. 

The student will accurately leam the best known works 
of Voltaire. 
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Tho teachor will help the class to solve algebra problems 
correctly. 

The student will appreciate the key Importance of algebraic 
'approaches. 

The student will include 10 supporting facts in a written 
"persuasive paragraph. 

The student will become familiar with how to write an 
’essay using no reference but personal experience. 
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INSTRUCTIONAL OBJECTIVES PREFERfNCE LIST Name 

Directions i Rate the Instructional objectives below according to 
the following schenei 
Excellenti £ Falri 2 

Good Poor* 1 

Average 3 

There are no "right" or "wrong" answers for this list, 
so please express your preferences candidly by placing 
a number before each objective. 

1 . The student will be able to comprehend thoroughly the ways 
in which our constitution permeates our every day life. 

2. When presented with a list of nouns and pronouns, the 
studont will be able to label each word correctly. 

3. Student will be able to see the value of reading the 
"classics" in his leisure time. 

b . The student will be able to write an essay alloying one 
of three logical organisations given in olass which exhibts 
no grammatical errors. 

5 . The student will be able to learn the number of voters in 
his preoinct. 

6 . The student will be able to list those articles in the 
constitution which relate to "due process of law." 

7 . Students will realise the importance of knowing the 
approximate date at which a given literary work was 
produced. 

8 . The teacher will cover the key tools of the chemistry 
lab, that is, the Bunse.i burner and various types of 
test tubes. 

9 . Oiven a list of 10 actual municipal court decisions, the 
student will be able to select the six which violate key 
tenets of the constitution and subsequently write an 
essay briefly explaining the nature of these violations. 

1 0. The student will orally recite the notes of six chemical 
compounds containing three or more elements. 
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lit The student will be able to cite some of the literary 
"classics" and briefly describe in an essay those 
features which give then universal appeal# 

12. The student will grasp the significance of oivlo re- 
sponsibility* 

13* The student will be able to name the date when women 
were first permitted to vote* 

lb. The teacher will discuss the grammatical form of the 
amendments to the constitution. 

li>» The student will be cognisant of the important role 
soientifio investigation has played in the field of 
chemistry and will become conversant with the relation- 
ship between scientific inquiry and the everyday life 
of the individual. 

16. The teacher will help the class to become proficient 
corammicators in written English* 

17* Oiven the names of veil- know novels and the names of 
contemporary authors, the student will be able to 
correctly match them in a test* 

18. The student will be able to write an essay in which he 
contrasts the arguments for having a democracy or total- 
itarian state. 

19* The student will learn the parts of speech. 

( 20. The student will be capable of setting up an experimental 
hypothesis test in the field of quantitative chemical 
analysis so that presented with an unknovn chemical com- 
pound he can thereafter correctly identify its consti- 
tuent elements* 



Scoring Instructions! 

Step 1. Add nuabers by items 2, b, 6, 9, 10, 11, 13, 17, 18, 
20. This is subtotal No. 1. 

Step 2. Add numbers by the remaining items. This is subtotal 
Ko. 2. 

Step 3* Subtract subtotal No. 2 from 60. 

Step b* Add the .'esults of 3 to subtotal No. 1. This is the 
score for the IOPL. 
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ABSTRACT 

During tho summer, 1968, a six-week Insorvieo training Institute vas held 
at Duluth, Minnesota, for 45 experienced teachers from 13 secondary schools 
throughout tho United States, These schools are part of a 17-school ES 1 70 
Network bandod togothor for the purpose of developing innovative curricula for 
tho 19 r '0 t s. Tho purpose of tho Institute was to prepare tho 45 teachers in tho 
following aroasi writing behavioral objectives and developing "learning 
packages" to attain those objectives \ gaining skill in group dynamics and 
developing moro solf -understanding, openness, ability to deal with affectivo 
phenomena j becoming acquainted with educational simulation j gaining experience 
with a broadly-based humonitios approach to learning. 

Significant gains wore attained in moasurod ability to identify, ovaluato, 
and dovolop behavioral objectives, and in measured attitudes toward teaching and 
students, in tho direction of moro acceptance, understanding, rospect for students. 
Subjective, opon-onded evaluation by onrolleos vr ..<» generally positivo toward tho 
experience of living and working togothor and toward the alms of tho Ihstituto, 
but also somewhat critical of tho humanities and educational simulation aspects 
of tho program. Based on thoughtful critique by onrolleos, changes aro recom- 
mended for futuro institutes, and a reunion followup is advlsod to pornit further 
evaluation and sharing of transfer values and oxporioncos. 



